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ABSTRACT 


The current controversy about using results as a way 



of assessing the success of efforts to improve the lives of children 
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system, has given rise to hope, fear, and confusion. This paper sets 
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and identifies a startup list of outcome measures with annotations on 
their use. The first section describes what is at stake in the shift 
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such a move. The second section discusses the risks, which can be 
recognized and minimized, involved in making the shift. Community 
goals and their relationship to resul t s~bas ed account abi 1 i ty are 
discussed in section 3. Guidelines for starting the shift to 
resul ts-based account ab i 1 i ty are highlighted in the fourth section. 
The fifth section provides an overview of the new issues raised by 
resul ts-based accountab i 1 i ty , including who decides on the desired 
outcomes, who is responsible for achieving those outcomes, the goal 
of process measures, the role of cost-savings measures, and the 
relationship between resul ts-based accountability and evaluation 
research. Next steps for resu 1 t s-based accountabi 1 i ty include 
continuing work to make a startup list more useful; new work to 
increase the significance of what is being measured; and new work to 
identify intermediate indicators. Products available from the 
Improved Outcomes for Children Project, which works with states and 
local communities undertaking reform of educational and human 
services systems, are also listed. (LMl) 
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The Case for Shifting to Results-Based Accountability 



T he current ferment around using re- 
sults as a way of assessing the suc- 
cess of efforts to improve the lives 
of children and families, and of 
shifting to a results-based account- 
ability system, has given rise to hope, fear, and 
confusion. 

Hope that results-based accountability could 
be the key to 

• freeing schools, health care, social agencies, 
and other human services from the rules that 
prevent them from operating flexibly in re- 
sponse to the needs of those they serve, 

• restoring the public's faith that both public 
and private human service institutions can ac- 
complish their intended purpose 

• encouraging communities to be more planful, 
more intentional, in how they support chil- 
dren and families. 

Fear that results-based accountability will be 
misused and bring about 

• the abandonment of attempts to better the 
conditions of disadvantaged children whose 
effects are difficult to measure or take a long 
time to occur 

• the erosion of essential procedural protections 
and neglect of concerns for equity 



• a smokescreen behind which further funding 
cutbacks will be made 

• penalizing individual professionals, institu- 
tions, and agencies who may not be achieving 
hoped-for results, but are trying hard and do- 
ing the best they can. 

Confusion about how to devise and obtain 
agreement on a set of goals and outcomes, and 
reliable ways of measuring results that could jus- 
tify the hopes and quell the fears of the many 
concerned constituencies. 

This paper sets out some of the issues in the 
shift to results-based accountability, and identi- 
fies a start-up list of outcome measures with an- 
notations on their use. The authors see results- 
based accountability as an essential part of a 
larger strategy to improve outcomes for chil- 
dren. Subsequent materials from the Improved 
Outcomes for Children Project will address how 
these outcomes can be translated into a program 
agenda; how that program agenda can then lead 
to a budget and a financial plan; and how, over 
time, results-based accountability can be com- 
bined with both rewards and penalties, based on 
performance. This paper sets the stage for those 
later discussions, but all readers, especially poli- 
cymakers, should note that the real benefit of re- 
sults-based accountability can be realized only 
through that full sequence of activities. 
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I 

What is at Stake In the Shift Toward 
Results-Based Accountability 



A. Results-based accountability can replace — or 
at least diminish the need for — centralized bu- 
reaucratic micromanagement and rigid rules. Ef- 
fective services require a significant degree of 
both local variation and frontline discretion, 
which cannot be maintained in the face of de- 
tailed regulation of program inputs that tie the 
hands of front-line professionals. Management 
by results is the best alternative to top-down, 
centralized micromanagement, which holds pro- 
grams responsible for adhering to rules that are 
so detailed that they interfere with a program’s 
or institution’s ability to respond to a wide range 
of urgent needs. 

Policy makers can be encouraged to desist 
from regulating inputs and prescribing detailed 
procedures if they have the capacity to hold pro- 
grams, institutions, and those who implement 
and run them accountable for results. The use of 
outcome indicators that reflect common sense 
and common understanding (indicators that 
show, for example, that the rates of low-weight 
births are being reduced, or that more students 
are demonstrating age-appropriate mastery of 
school skills) helps to focus attention on agency 
mission rather than rules, and permits the neces- 
sary flexibility and autonomy at the front-end. 
Auditors can spend less time reviewing records 
to see how many services were provided (e.g. 
how many families were seen) and whether the 
forms documenting eligibility for services were 
properly filled out. Instead, they spend more 
time on inquiring into the results achieved (such 
as the reduction of multiple or inappropriate 



out-of-home placements for children.) The ques- 
tion asked of professionals at the front-lines, be 
they teachers, social workers, or health profes- 
sionals, would shift from “Did you do what they 
told you to do?” to “Did it work? What differ- 
ence did it make in outcomes for children?” A 
changed organizational climate would be the re- 
sult, in which well-trained professionals would 
be able to use their judgment and experience to 
respond flexibly to the needs of children and 
families, rather than being constrained by pres- 
sures of funders who see detailed input regula- 
tion as their only means of protecting the public 
interest. 

One example of how this approach can work 
comes from Kentucky: under the Kentucky Edu- 
cation Reform Act, the state decides what, and 
local communities decide how. (The process of 
deciding what a state will hold communities ac- 
countable for will optimally involve extensive 
consultation, as we discuss in Section V.A. on 
page 13). 

B. Outcome information can assure funders and 
the public that investments are producing re- 
sults. Funders and the public are demanding in- 
formation on which informed judgments can be 
made about whether institutions, programs and 
policies are in fact accomplishing their intended 
purpose. Polls show that a public that has lost 
confidence in government and other institutions 
is prepared to support new investments in 
schools and services when citizens are convinced 
that the investment is bringing promised results. 
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The Case for Shifting to Results-Based Accountability 



Especially in a time of fiscal constraints, pro- 
grams and agencies willing to be held account- 
able for achieving agreed-upon results will have 
a greater chance of obtaining needed funding 
and other support. 

C. Agreement on desired results can facilitate 
cross-systems collaboration on behalf of children. 
As more individuals and agencies from different 
systems, disciplines, and backgrounds attempt to 
work together, they encounter barriers that are 
difficult to overcome in the absence of shared 
purposes that are explicitly defined and agreed 
upon. Agreement on a common set of goals and 
outcome measures not only makes collaboration 
easieg but also helps promote a community-wide 
“culture of responsibility” for children and fami- 
lies, and fuels the momentum for change. 

D. Agreement on desired results helps to mini- 
mize investment in activities that don’t con- 
tribute to improved results. Reflecting Alice in 
Wonderland’s insight that if you don’t know 
where you’re going, any road will get you there, 
a focus on outcomes is likely to discourage ex- 
penditures of energy, political capital and funds 
on empty organizational changes and on ineffec- 
tive services. Community-wide acceptance of 
shared outcomes thus helps to put service inte- 
gration efforts into proper perspective. Fragmen- 
tation is such a big problem in existing services, 
that a great deal of energy is currently going into 
efforts to link services together. But while collab- 
oration is a necessary condition for providing ef- 
fective services, it is not sufficient. Alone, it will 
not improve results. It is futile to put together 
services that are ineffective because they are of 
mediocre quality, are rendered grudgingly, are 
rendered by professionals who don’t work re- 
spectfully with families, and are unable to re- 
spond to the unique characteristics of the com- 
munity they serve. The shared commitment to 



improve outcomes for children is what can make 
efforts at collaboration fall into place — not as an 
end, but as an essential means of working to- 
gether toward improved results. 

E. Information about results enhances commu- 
nity and agency capacity to judge the effective- 
ness of their efforts, and to modify activities in 
response to impact information. Because this is a 
time of rapid change in education, welfare, 
health and human services practices, programs, 
and systems, it is particularly important to im- 
prove the ability to make judgments about the 
effects of change: whether new or reformed ser- 
vices, new relationships among service providers 
and between providers and communities, and 
new connections among schools, services, and 
other community and economic development ef- 
forts, are working and worth the investment. 
When results are tracked on a timely basis and 
connected to service strategies, outcome infor- 
mation can provide guidance to program devel- 
opment as initiatives evolve. 

F. A focus on results clarifies whether allocated 
resources are adequate to achieve the outcomes 
expected by funders and the public. An out- 
comes focus highlights how much investment is 
required to produce significant improvement in 
the lives of children and families. The new con- 
versation about results is promising (or threaten- 
ing) to end a conspiracy of silence between fun- 
ders and program people. It exposes the fact 
that human service providers, educators, and 
community organizations are consistently ex- 
pected to accomplish massive tasks with trivial 
resources and inadequate tools. It assures atten- 
tion to the adequacy question: whether the re- 
sources being committed are adequate to 
achieve the intended purpose. 

A parent education program may be expected 
to reduce the incidence of child abuse in a neigh- 




7 



5 



borhood although it consists of only a few di- 
dactic classes. An outreach program to get preg- 
nant women into prenatal care may be expected 
to reduce the incidence of low birth weight in a 
community although the sources of prenatal 
care are overcrowded, impersonal, and have no 
capacity to deal with homelessness, drug abuse, 
or lack of social support. In the past, when a sig- 
nificant discrepancy occurred between aspira- 
tions and documentable (or even potential) ac- 
complishments, the response has typically been 
to retreat from a commitment to changing out- 
comes. As program managers and funders face 
up to the difficulties of actually changing real- 
world outcomes, and recognize the relative 
weakness of single, underfunded, piecemeal in- 
terventions compared to powerful social, eco- 
nomic, and demographic forces that push out- 
comes in a negative direction, they often agree 
just to measure how many people were reached 
or how attitudes and knowledge have changed 
among program participants. Together, all par- 
ties agree that it is unfair or inappropriate to 
hold the program accountable for actually im- 
proving results when the program is doing the 



best it can. The reality that the investment is in- 
sufficient is thereby obscured. 

In circumstances where it will take a critical 
mass of comprehensive, intensive, interactive in- 
terventions to change outcomes in a defined 
population, where interventions must be able to 
impact even widespread despair, hopelessness 
and social isolation in order to be effective, it is 
counterproductive to hide from ourselves the 
limitations of most current efforts. Recognition 
that the addition of a single new intervention, or 
even recognition that services alone may not be 
sufficient to change outcomes in the absence of 
changes in employment, economic security, phys- 
ical safety and housing, is not an argument 
against results-based accountability. It is an argu- 
ment for adequate funding of promising interven- 
tions, and an argument for human service agen- 
cies and schools to forge partnerships with efforts 
to create jobs and to improve housing and public 
safety, and to re-create the infrastructure of com- 
munities. It is an argument for defining what the 
community considers a set of adequate outcomes 
and then making sure that the resources needed 
to reach those results are made available. 




n 

Risks— to Recognize and to Minimize— 
in Making the Shift to 
Results-based Accountability 




While the shift toward results-based account- 
ability brings many clear advantages, it is not an 
unmixed blessing. In fact, it carries real risks, 
which must be recognized and minimized 
through careful strategizing and thoughtful im- 
plementation. 



A. Funders, the public, and even program people, 
may underestimate how long it takes to achieve 
significant improvement in outcomes. The single 
most frequently cited lesson from major current 
reform efforts is that it takes much more time 
than expected — both to get the initiative under 
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way, and to get it to the point where it begins to 
show an impact on real-world outcomes. 

B. Demands for documented results could 
drive programs away from achieving their mis- 
sion of improving a broad range of results for a 
broad target population, when agencies 

• engage in “creaming” and ducking hard 
cases 

• distort their activities to emphasize those that 
will show measurable and rapid results while 
neglecting equally or more important efforts 
that are harder to quantify and whose results 
don’t show up as quickly. (For example, if im- 
munization rates are the outcome on which 
health services are judged, it will be important 
to guard against allowing an emphasis on im- 
munization strategies to result in the neglect 
of other primary and preventive services.) 

C. Funders could be led to confine their support 
to interventions whose effects are readily and 
quickly quantifiable, in preference to more effec- 
tive or subtler interventions whose benefits are 
harder to document. (A church-sponsored chil- 
dren’s choir or a recreation center for teenagers 
may add to a sense of community, and thereby 
contribute to improved outcomes, without being 
able to demonstrate a direct outcome impact.) 
Protection must be explicitly provided to inter- 
ventions that are vulnerable because 

• their effects show up years later 

• the data to document their impact are not 
available, or the technology to measure their 
effects does not exist 

• the intervention is one of many preventive ef- 
forts that interact to produce cumulative later 
effects. 

D. Some forms of outcome measurement could 
lead to labeling and stigmatizing children. Ef- 



forts to assess school readiness, for example, 
must be designed to take into account that 
“readiness” does not inhere solely in the child, 
that children don’t all develop at the same rate, 
and that “readiness” is not just a cognitive mat- 
ter, but includes physical and emotional health 
and social competence. In addition, school readi- 
ness should be assessed on a sample rather than 
an individual basis, to avoid labeling of individ- 
ual children. 

E. Agency accountability could be weakened as 
attention shifts to community-wide accountabili- 
ty efforts. Achieving any of the core outcomes 
requires the activities of multiple agencies, as 
well as informal supports and community activi- 
ties that are beyond a single agency’s control. 
Since no one agency, acting alone, can achieve 
any of the agreed-upon results, the question aris- 
es, how should individual public agencies, or 
systems — for example, the child welfare agency 
or the mental health system — hold themselves or 
be held accountable. 

The answer lies in a careful “unbundling” of 
each outcome into the detailed steps and strate- 
gies which, together, can produce the intended re- 
sult. For example, to increase healthy births, a 
host of related actions are necessary, one of 
which is likely to be that the local health depart- 
ment must organize and conduct outreach to all 
high-risk mothers to ensure that they have ade- 
quate prenatal care. Once the health depart- 
ment’s role is specified within the context of the 
overall strategy, measures can be applied to how 
well the health department is performing this role 
and the department can be held (and can hold it- 
self) accountable for this result. So, to continue 
this example, if all parts of a strategy to increase 
healthy births are successfully accomplished ex- 
cept one (e.g., the health department’s), then the 
health department can be held accountable by 
the community for better performance. 
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F. The shift to results-based accountability 
should not be seen as a panacea. An outcomes 
orientation does not guarantee the design of an 
effective community service and support system. 
Especially because current understanding of the 
precise connections between specific interven- 
tions and specific outcomes is limited, an out- 
comes orientation does not solve the problem of 
identifying needed and missing elements of a 
comprehensive approach to improving results. 
However, by informing and focusing the process, 
result-based thinking does seem to support the 
development of a culture that is less rule-bound 
and more mission-driven. 

G. The shift to results-based accountability can- 
not be allowed to substitute for rock-bottom 
safeguards against fraud, abuse, poor services, 
and inequities or discrimination based on race, 
gender, disability or ethnic background. Sup- 
porters of improved outcomes for children may 
in the past have overestimated the extent to 
which equity and quality could be safeguarded 
by regulation, and underestimated how much 
detailed mandating and strict rules could under- 
mine responsiveness, flexibility, and the exercise 
of discretion. If the shift to results-based ac- 
countability is to attain its promise, it must be 
combined with better training of front-line 
workers and managers and careful monitoring, 
to make the achievement of less rigidity in the 
provision of services compatible with the protec- 
tion of vulnerable children and families. 

The new outcomes orientation should not 
lead to the abandonment of all input and 
process regulation. Procedural protections will 



have to be maintained to protect against fraud, 
poor services, and inequities or discrimination 
based on race, gender, disability, or ethnic back- 
ground. Procedural protections will also have to 
be maintained where there is no other way to re- 
strict the arbitrary exercise of front-line discre- 
tion by powerful institutions against the interests 
of powerless clients. At the same time, reliance 
on process measures should not be allowed to 
drive out or obscure a continuing focus on re- 
sults-based accountability. 

H. When a community adopts a results-orient- 
ed system, fundamental changes in the alloca- 
tion of resources will result and must be pre- 
pared for. One key indicator of whether the 
outcomes orientation is influencing agency per- 
formance will be whether budget priorities 
change in response to the identification of 
shared outcome measures. The shift to results- 
based accountability will be of little use in a 
community if it does not translate into manage- 
ment and budget priorities. The adoption of 
community outcomes will be ineffective if agen- 
cies revert to tradition and divide the results in- 
to isolated subsets that would be pursued sepa- 
rately by schools, social service agencies, the 
health sector, mental health agencies, juvenile 
justice, and others. As agencies, communities 
and systems begin to manage their resources to 
achieve agreed-upon results, set priorities based 
on these outcomes, and are held accountable for 
making progress toward these outcomes, some 
agencies will actually lose resources in the 
process. That risk must be acknowledged and 
prepared for at the outset. 
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m 

Community Goals and their Relationship 
to Results-Based Accountability 



Communities, states, and collaboratives can best 
begin the results-based accountability process by 
asking the question, What do we want for our 
children? By beginning in this Way, these deliber- 
ations are usually able to go beyond earlier pro- 
ject — or problem-bounded discussions. They are 
then more likely to produce a consensus around 
basic goals which together constitute a vision 
that can serve to: 

• capture aspirations 

• unify people who differ in many other ways 
around common aims for their children 

• focus attention not only on a wide range of 
formal services, but also on informal supports 
and opportunities 

• surface value issues in the context of what 
people want for their children, before they get 
into the question of what is to be measured. 

Examples of the goals that communities might 
choose include 

• higher rates of healthy births 

• higher rates of children ready for school 
learning at the time of school entry 

• higher rates of children succeeding at school 

• higher rates of youth who achieve personal, 
social, and vocational competence (which 
might be further broken down into higher 
rates of youth who feel safe, who have a sense 
of self-worth, a sense of mastery, a sense of 
belonging, a sense of personal efficacy, who 
are socially, academically, and culturally com- 



O 




petent, and who have the skills needed for 

productive employment.) 

As a community embarks on the job of building 
an effective system of services and supports, 
these goals can become a framework that can 
be used for many purposes — inspirational, 
guiding service delivery, and guiding the selec- 
tion of outcome measures for accountability 
purposes. 

Only some aspects of these goals can current- 
ly be measured with widely available data and 
with outcome measures around which it is pos- 
sible to gain widespread agreement. Most com- 
munities will have aspirations for their children 
that exceed the results that are currently mea- 
surable. Goals and outcome measures serve dif- 
ferent purposes. The former represent what the 
community is striving for. The latter represent 
what the community will be held accountable 
for — by public and private funders and perhaps 
by higher levels of government. The goals can 
be general, but the outcome measures must be 
so specific, the public stake in their attainment 
so clear, and their validity and reliability so well 
established, that the community would be will- 
ing to see rewards and penalties, as well as re- 



1 K. Pittman, S. Zeldin, “Evaluating Youth Development 
in Programs and C ommunities: The Need for an Integrat- 
ed Framework and Collaborative Strategy.” Washington, 
DC: Center for Youth Development and Policy Research. 
1992 
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